Memory Hierarchy Design for Caching Middleware in the Age of NVM

Advances in storage technology have introduced Non-Volatile Memory, NVM, as a new storage medium. NVM, along with DRAM, NAND Flash, and Disk present a system designer with a wide array of options in designing caching middleware. This paper provides a systematic way to use knowledge about the frequencies of read and write requests to individual data items in order to determine the optimal cache configuration given a fixed budget. The approach introduced in this paper incorporates the characteristics of each type of memory to answer key design questions such as: how much will an increase in budget improve expected retrieval/update times? If it is desirable to limit the number of different types of memory in a cache, then which types of memory should be used for a particular budget and database size? When is it advantageous to store multiple copies of data objects in order to quickly recover from a memory failure? The cache configuration problem is modeled as an instance of the Multiple Choice Knapsack Problem (MCKP). Although MCKP is NP-complete, its linear programming relaxation is efficiently solvable and can be used to closely approximate the optimal solution. We use an algorithm for MCKP to evaluate design trade-offs in the context of a memory hierarchy for a Key-Value Store (e.g., memcached) as well as a host-side cache (e.g., Flashcache) to store disk pages. The results show selective replication is appropriate with certain failure rates. With a slim failure rate, tiering of data across the different storage media that constitute the cache is superior to replication.